Enterprise storage systems currently available are proprietary storage appliances that integrate the storage controller functions and the storage media into the same physical unit. This centralized model makes it harder to independently scale the storage systems' capacity, performance and cost. Users can get tied to one expensive appliance without the flexibility of adapting it to different application requirements that may change over time. For small and medium scale enterprise, this may require huge upfront capital cost. For larger enterprise datacenters, new storage appliances are added as the storage capacity and performance requirements increase. These operate in silos and impose significant management overheads.
These enterprise storage systems can store data as key-value pairs, where a key is a reference for the data and the value is the data to be stored. Current implementation of a key-value pairs, however, do not lend themselves to supporting multiple data-sets each backed by and independent policy varying in availability, performance, storage placement, storage format, indexing, and consistency guarantees. In addition, a key-value pair does not handle supporting virtualization that separates data sets and their physical location mapping. Such mechanisms, if it exists, would apply to all data in the system (e.g., selective data sets cannot be moved around). In addition, there is no implementation that provides for named/unnamed writes that co-exists in the same system.
Furthermore, current enterprise storage systems implementations do not optimize on key-value cardinality properties. For example, the system were to use an “existing” key values store, that system will not be able to optimize on the cardinality property of the system's use case (using a key value store to build a filesystem.) Thus, if keys have one to one mapping to values, certain operations pertaining to consistency, availability and repair can be heavily simplified and optimized. However, if the keys have a one-to-many relationship to values, then extra effort has to be made, to provide consistency availability and repair guarantees. In addition, a log system, that has insight about the key being accessed has a one-to-one relationship with a value, can provide fast path access to keys without going through reconcile step required for keys with one-to-many relationships.